{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.1、Model representation\n",
    "\n",
    "#### 1. supervised learning:\n",
    "\n",
    "1. Regression Problem\n",
    "\n",
    "2. Classification Problem\n",
    "\n",
    "#### 2. Notation:\n",
    "\n",
    "> m = Number of training examples\n",
    ">\n",
    "> x = \"input\" variable\n",
    ">\n",
    "> y = \"output\" variable\n",
    ">\n",
    "> (x, y) = one training example\n",
    ">\n",
    "> $(x^{(i)}, y^{(i)})$ = $i^{th}$ training example\n",
    "\n",
    "#### 3. How to work:\n",
    "\n",
    "![](../pic/QQ截图20201020183511.png)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.2、Cost function\n",
    "\n",
    "* eg1:\n",
    "\n",
    "![](../pic/QQ截图20201020210900.png)\n",
    "\n",
    "\n",
    "![](../pic/QQ截图20201020211522.png)\n",
    "\n",
    "* define:\n",
    "\n",
    "$$\n",
    "Hypothesis:\\quad h_{\\theta}(x)=\\theta_{0}+\\theta_{1}x\\\\\n",
    "Parameters:\\quad \\theta_{0}, \\theta_{1}\\\\\n",
    "Cost Function:\\quad J(\\theta_{0}, \\theta_{1})=\\frac{1}{2m}\\sum_{i=1}^{m}(h_{\\theta}(x^{(i)})-y^{(i)})^2\\\\\n",
    "Goal:\\quad \\min_{(\\theta_{0},\\theta_{1})}J(\\theta_{0}, \\theta_{1})\n",
    "$$\n",
    "\n",
    "* assert: $\\theta_{0}=0$\n",
    "* 2 dimension\n",
    "* compute the cost\n",
    "\n",
    "![](../pic/QQ截图20201020214313.png)\n",
    "\n",
    "* assert: $\\theta_{0}\\neq 0$\n",
    "* 3 dimension\n",
    "* compute the cost\n",
    "\n",
    "![](../pic/QQ截图20201020215444.png)\n",
    "\n",
    "![](../pic/QQ截图20201020215957.png)\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.5、Gradient descent\n",
    "\n",
    "* setup:\n",
    "\n",
    "> Have: function $J(\\theta_{0}, \\theta_{1})$\n",
    ">\n",
    "> Want: $\\min_{(\\theta_{0},\\theta_{1})}J(\\theta_{0}, \\theta_{1})$\n",
    "\n",
    "* outline:\n",
    "   * start with some $\\theta_{0}, \\theta_{1}$\n",
    "   * change $\\theta_{0}, \\theta_{1}$ to reduce $J(\\theta_{0}, \\theta_{1})$ until min\n",
    "   \n",
    "* downhill:\n",
    "\n",
    "![](../pic/QQ截图20201020221113.png)\n",
    "\n",
    "* Gradient descent algorithm\n",
    "\n",
    "$$\n",
    "{\\alpha} : learning rate\n",
    "$$\n",
    "\n",
    "![](../pic/QQ截图20201020221914.png)\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.6、Gradient descent intuition\n",
    "\n",
    "* algorithm\n",
    "\n",
    "$\n",
    "\\quad repeat until convergence\\{ \\\\ \n",
    "\\quad\\quad\\theta_{j}:=\\theta_{j}-\\alpha \\frac{\\partial}{\\partial\\theta_{j}}J(\\theta_{0}, \\theta_{1})\\quad (j=1,2,...n) \\\\ \n",
    "\\quad\\} \\\\ \n",
    "$ \n",
    "\n",
    "* explain:\n",
    "\n",
    "![](../pic/QQ截图20201021090002.png)\n",
    "\n",
    "![](../pic/QQ截图20201021090447.png)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.7、Gradient descent for linear regression\n",
    "\n",
    "* \"Batch\"\n",
    "* explain:\n",
    "\n",
    "$$\n",
    "\\frac{\\partial}{\\partial\\theta_{j}}J(\\theta_{0}, \\theta_{1})=\\frac{\\partial}{\\partial\\theta_{j}}\\frac{1}{2m}\\sum_{i=1}^{m}(h_{\\theta}(x^{(i)})-y^{(i)})^2\\\\\n",
    "\\quad=\\frac{\\partial}{\\partial\\theta_{j}}\\frac{1}{2m}\\sum_{i=1}^{m}(\\theta_{0}+\\theta_{1}x^{(i)}-y^{(i)})^2\\\\\n",
    "$$\n",
    "\n",
    "* so:\n",
    "\n",
    "$$\n",
    "j=0:\\frac{\\partial}{\\partial\\theta_{0}}J(\\theta_{0}, \\theta_{1})=\\frac{1}{m}\\sum_{i=1}^{m}(h_{\\theta}(x^{(i)})-y^{(i)})\\\\\n",
    "j=1:\\frac{\\partial}{\\partial\\theta_{1}}J(\\theta_{0}, \\theta_{1})=\\frac{1}{m}\\sum_{i=1}^{m}(h_{\\theta}(x^{(i)})-y^{(i)}) x^{(i)}\\\\\n",
    "$$\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
